Convergence Rate Research Articles

Improve robustness of machine learning via efficient optimization and conformal prediction

AbstractThe advance of machine learning (ML) systems in real‐world scenarios usually expects safe deployment in high‐stake applications (e.g., medical diagnosis) for critical decision‐making process. To this end, provable robustness of ML is usually required to measure and understand how reliable the deployed ML system is and how trustworthy their predictions can be. Many studies have been done to enhance the robustness in recent years from different angles, such as variance‐regularized robust objective functions and conformal prediction (CP) for uncertainty quantification on testing data. Although these tools provably improve the robustness of ML model, there is still an inevitable gap to integrate them into an end‐to‐end deployment. For example, robust objectives usually require carefully designed optimization algorithms, while CP treats ML models as black boxes. This paper is a brief introduction to our recent research focusing on filling this gap. Specifically, for learning robust objectives, we designed sample‐efficient stochastic optimization algorithms that achieves the optimal (or faster compared to existing algorithms) convergence rates. Moreover, for CP‐based uncertainty quantification, we established a framework to analyze the expected prediction set size (smaller size means more efficiency) of CP methods in both standard and adversarial settings. This paper elaborates the key challenges and our exploration towards efficient algorithms with details of background methods, notions for robustness measure, concepts of algorithmic efficiency, our proposed algorithms and results. All of them further motivate our future research on risk‐aware ML that can be critical for AI–human collaborative systems. The future work mainly targets designing conformal robust objectives and their efficient optimization algorithms.

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

BackgroundA Generalized Linear Mixed Model (GLMM) is recommended to meta-analyze diagnostic test accuracy studies (DTAs) based on aggregate or individual participant data. Since a GLMM does not have a closed-form likelihood function or parameter solutions, computational methods are conventionally used to approximate the likelihoods and obtain parameter estimates. The most commonly used computational methods are the Iteratively Reweighted Least Squares (IRLS), the Laplace approximation (LA), and the Adaptive Gauss-Hermite quadrature (AGHQ). Despite being widely used, it has not been clear how these computational methods compare and perform in the context of an aggregate data meta-analysis (ADMA) of DTAs.MethodsWe compared and evaluated the performance of three commonly used computational methods for GLMM - the IRLS, the LA, and the AGHQ, via a comprehensive simulation study and real-life data examples, in the context of an ADMA of DTAs. By varying several parameters in our simulations, we assessed the performance of the three methods in terms of bias, root mean squared error, confidence interval (CI) width, coverage of the 95% CI, convergence rate, and computational speed.ResultsFor most of the scenarios, especially when the meta-analytic data were not sparse (i.e., there were no or negligible studies with perfect diagnosis), the three computational methods were comparable for the estimation of sensitivity and specificity. However, the LA had the largest bias and root mean squared error for pooled sensitivity and specificity when the meta-analytic data were sparse. Moreover, the AGHQ took a longer computational time to converge relative to the other two methods, although it had the best convergence rate.ConclusionsWe recommend practitioners and researchers carefully choose an appropriate computational algorithm when fitting a GLMM to an ADMA of DTAs. We do not recommend the LA for sparse meta-analytic data sets. However, either the AGHQ or the IRLS can be used regardless of the characteristics of the meta-analytic data.

Online identification and feed-forward compensation of nonlinear friction in servo system based on RBF neural network model

In this paper, an online identification and compensation method of nonlinear friction based on radial basis function (RBF) neural network model is proposed for the influence of nonlinear friction on machining accuracy in the low speed process of servo feed system of CNC machine tools. First, a three-layer single-input-output RBF neural network model is established for describing the nonlinear friction of servo feeding system. Second, the neural network online learning algorithm is improved based on adaptive gain, which improves the stability and accuracy of the algorithm. Finally, experiments were carried out on a three-axis milling machine to compensate the friction in the servo feed system in real time based on the online identification results. The results show that the method can effectively improve the online identification accuracy and convergence rate, and effectively improved the low-speed performance of the servo feed system.

A note on ideal C$$^*$$-completions and amenability

AbstractFor a discrete group G, we consider certain ideals $$\mathcal {I}\subset c_0(G)$$ I ⊂ c 0 ( G ) of sequences with prescribed rate of convergence to zero. We show that the equality between the full group C$$^*$$ ∗ -algebra of G and the C$$^*$$ ∗ -completion $$\textrm{C}^*_{\mathcal {I}}(G)$$ C I ∗ ( G ) in the sense of Brown and Guentner (Bull. London Math. Soc. 45:1181–1193, 2013) implies that G is amenable.

Uniqueness and numerical inversion in bioluminescence tomography with time-dependent boundary measurement

Abstract In the paper, an inverse source problem in bioluminescence tomography (BLT) is investigated. BLT is a method of light imaging and offers many advantages such as sensitivity, cost-effectiveness, high signal-to-noise ratio and non-destructivity. It thus has promising prospects for many applications such as cancer diagnosis, drug discovery and development as well as gene therapies. In the literature, BLT is extensively studied based on the (stationary) diffusion approximation (DA) equation, where the distribution of peak sources is reconstructed and no solution uniqueness is guaranteed without proper a priori information. In this work, motivated by solution uniqueness, a novel dynamic coupled DA model is proposed. Theoretical analysis including the well-posedness of the forward problem and the solution uniqueness of the inverse problem are given. Based on the new model, iterative inversion algorithms under the framework of regularizing schemes are introduced and applied to reconstruct the smooth and non-smooth sources. We discretize the regularization functional with the finite element method and give the convergence rate of numerical solutions. Several numerical examples are implemented to validate the effectiveness of the new model and the proposed algorithms.

Slime Mould Algorithm Based on a Gaussian Mutation for Solving Constrained Optimization Problems

The slime mould algorithm may not be enough and tends to trap into local optima, low population diversity, and suffers insufficient exploitation when real-world optimization problems become more complex. To overcome the limitations of SMA, the Gaussian mutation (GM) with a novel strategy is proposed to enhance SMA and it is named as SMA-GM. The GM is used to increase population diversity, which helps SMA come out of local optima and retain a robust local search capability. Additionally, the oscillatory parameter is updated and incorporated with GM to set the balance between exploration and exploitation. By using a greedy selection technique, this study retains an optimal slime mould position while ensuring the algorithm’s rapid convergence. The SMA-GM performance was evaluated by using unconstrained, constrained, and CEC2022 benchmark functions. The results show that the proposed SMA-GM has a more robust capacity for global search, improved stability, a faster rate of convergence, and the ability to solve constrained optimization problems. Additionally, the Wilcoxon rank sum test illustrates that there is a significant difference between the optimization outcomes of SMA-GM and each compared algorithm. Furthermore, the engineering problem such as industrial refrigeration system (IRS), optimal operation of the alkylation unit problem, welded beam and tension/compression spring design problem are solved, and results prove that the proposed algorithm has a better optimization efficiency to reach the optimum value.

Parametric Bernstein operators based on contagion distribution

This paper proposes the extended form of the ‐Bernstein operators with the Pólya–Eggenberger distribution, also known as contagion distribution, using the integral operators for . We will refer to these sequence of operators as the “Parametric‐Bernstein operators based on contagion distribution” or simply the modified form of ‐Bernstein operators. These operators are defined with the help of two parameters, namely, and . We observe the significance of both these parameters. Subsequently, we calculate their moments and on the basis of Korovkin's approximation theorem we assert that our defined operators converge for a real valued continuous function . We give some basic properties, study the Voronovskaya type asymptotic result and discuss the ‐statistical approximation of our operators. We also use the modulus of continuity to study the rate of convergence. Alongside our primary work, we have also defined the King‐type modification that preserves the operators at and . Further, using modulus of continuity, we compare the properties and behavior of both the operators.

Stokes problem with slip boundary conditions using stabilized finite elements combined with Nitsche

We discuss how slip conditions for the Stokes equation can be handled using Nitsche method, for a stabilized finite element discretization. Emphasis is made on the interplay between stabilization and Nitsche terms. Well-posedness of the discrete problem and optimal convergence rates, in natural norm for the velocity and the pressure, are established, and illustrated with various numerical experiments. The proposed method fits naturally in the context of a finite element implementation while being accurate, and allows an increased flexibility in the choice of the finite element pairs.

Automatic calibration of a discrete element model of a masonry arch by swarm intelligence methods

This paper addresses the automatic calibration of constitutive parameters of the Discrete Element Method (DEM) models of masonry structures. Three swarm intelligence algorithms were applied, which are efficient in problems characterised by the small number of unknown parameters but high computational demand for the evaluation of the target function. Trust-Based Particle Swarm Optimisation (TBPSO) was chosen due to its excellent convergence rate. It was already tested that TBPSO converged faster than the original Genetic Algorithm and Particle Swarm Optimisation Methods. Therefore, TBPSO was compared with two recent and promising methods, the Marine Predator Algorithm (MPA) and the Golden Eagle Optimiser (GEO). The paper also presents a novel two-step approach, which enhances the performance of the tested optimisation strategies. The methodology is capable to account for and precisely find the mode of failure using Convolutional Neural Networks (CNN). The three optimisation methods and the two-step approach were applied on the DEM model of a quasi-static masonry arch collapse experiment from the literature. Performance of the three algorithms in identifying the appropriate constitutive parameters values of the DEM model were compared. Clear evidence on the advantage of the two-step approach with TBPSO is demonstrated. Remarks about different modelling issues are also included.

Estimating Markov Chain Mixing Times: Convergence Rate Towards Equilibrium of a Stochastic Process Traffic Assignment Model

Network equilibrium models have been extensively used for decades. The rationale for using equilibrium as a predictor is essentially that (i) a unique and globally stable equilibrium point is guaranteed to exist and (ii) the transient period over which a system adapts to a change is sufficiently short in time that it can be neglected. However, we find transport problems without a unique and stable equilibrium in the literature. Even if it exists, it is not certain how long it takes for the system to reach an equilibrium point after an external shock onto the transport system, such as infrastructure improvement and damage by a disaster. The day-to-day adjustment process must be analysed to answer these questions. Among several models, the Markov chain approach has been claimed to be the most general and flexible. It is also advantageous as a unique stationary distribution is guaranteed in mild conditions, even when a unique and stable equilibrium does not exist. In the present paper, we first aim to develop a methodology for estimating the Markov chain mixing time (MCMT), a worst-case assessment of the convergence time of a Markov chain to its stationary distribution. The main tools are coupling and aggregation, which enable us to analyse MCMTs in large-scale transport systems. Our second aim is to conduct a preliminary examination of the relationships between MCMTs and critical properties of the system, such as travellers’ sensitivity to differences in travel cost and the frequency of travellers’ revisions of their choices. Through analytical and numerical analyses, we found key relationships in a few transport problems, including those without a unique and stable equilibrium. We also showed that the proposed method, combined with coupling and aggregation, can be applied to larger transport models. History: This paper has been accepted for the Transportation Science Special Issue on the 25th International Symposium on Transportation and Traffic Theory. Funding: This study was financially supported by the Japan Society for the Promotion of Science [Grant-in-Aid 20H00265]. Supplemental Material: The online appendices are available at https://doi.org/10.1287/trsc.2024.0523 .

Deviation inequalities for dependent sequences with applications to strong approximations

In this paper, we give precise rates of convergence in the strong invariance principle for stationary sequences of bounded real-valued random variables satisfying weak dependence conditions. One of the main ingredients is a new Fuk-Nagaev type inequality for a class of weakly dependent sequences. We describe also several classes of processes to which our results apply.

An intelligent control method for a flexible-link manipulator with uncertain backlash and model uncertainty

This study examines a new intelligent control method for a single-link flexible manipulator that addresses backlash and model uncertainty. First, a smooth backlash inverse model is constructed to mitigate backlash nonlinearity. Subsequently, a ‘disturbance-like’ term is formulated to recharacterize the coupled term composed of external disturbances and model uncertainty. A new adaptive controller is proposed to compensate for the unknown ‘disturbance-like’ term. Using the proposed control method, the stability of the system is evaluated using the direct Lyapunov theory, ensuring uniform ultimate boundedness. Finally, numerical simulations and experiments are conducted using the Quanser platform. The numerical simulation and experimental results show that the proposed control can ensure a faster convergence rate and effectively reduce actuator input chattering.

Modal-aware prompt tuning with deep adaptive feature enhancement

Prompt learning has recently emerged as a promising method for fine-tuning vision-language models. By introducing prompts in the text encoder or image encoder, the pre-trained model can quickly adapt to downstream tasks without updating the pre-trained weights. However, prior multi-modal prompt tuning works do not consider the difference in feature distributions between text and images, and adopt the same prompts for both encoders, thus achieving sub-optimal performance in the downstream few-shot learning. In this paper, we propose Modal-Aware Prompt (MAP) to alleviate this issue. Specifically, considering the stability of text features, we design text-specific prompts, which can acquire text class-related information from a general template (i.e., “a photo of a <category>”) by unidirectional attention-based interaction. Additionally, considering the diversity of image features, we design visual-specific prompts to acquire image class-related information and adjust the image features by bidirectional attention-based interaction. To learn hierarchical prompt representations and reinforce the prompt features, we further propose a Deep Adaptive Feature Enhancement (DAFE) module to adaptively utilize the prompt output of the former layer, which can combine instance-level and task-level information simultaneously. Combining the above two designs, our method MAP-DAFE obtains state-of-the-art results on 11 image recognition datasets and has the fastest convergence rate. This proves our MAP-DAFE is effective and efficient.

Trajectory Tracking Algorithm Study of Coal Mine Water Detector Drilling Bar Installation

Mechanical water detection is recognized as the most reliable and safe production technology for coal mines, mainly for the detection of water hazards in pre-mining operations. Intelligent water detectors are currently the main research direction in mechanical water detection, and the automatic installation of drilling bars is the key to achieving intelligent water detection. Improving the connection accuracy in the process of installing drilling bars is an important research topic for the improvement of control links. To improve the connection accuracy of the drilling bars at the time of supplying material, we used the modified Denavit–Hartenberg method to analyze the motion gestures of the supplied material device and the Lagrange equation to establish a dynamic analysis model. We aimed at better control precision by improving the sliding mode control algorithm and at increasing the convergence rate of tracking errors with a sliding controller based on an exponential approximation law and using saturated functions instead of the symbol functions in the reaching law to weaken the vibration in the control process. We then used particle swarm optimization (PSO) to find the optimum combination parameters of the sliding mode controllers and test the performance of the sliding mode controllers before and after PSO with MATLAB/Simulink. The results showed that the optimized controller has a strong resistance to parameter fluctuations, and the system responds quickly, achieves a good performance, and improves the convergence rate of tracking errors.

A peridynamic-informed deep learning model for brittle damage prediction

Physics-informed Neural Network (PINN) has been introduced recently to predict and understand complex physical phenomena by directly incorporating feedback from governing equations. While PINNs offer remarkable capabilities, challenges arise when addressing discontinuities and heterogeneity, potentially compromising accuracy and reliability in prediction outcomes. In this study, a novel approach that combines the principles of peridynamic (PD) theory with PINN is presented to predict quasi-static damage and crack propagation in brittle materials. To achieve high prediction accuracy and convergence rate, the linearized PD governing equation is enforced in the PINN’s residual-based loss function. The proposed PD-INN is able to learn and capture complicated displacement patterns associated with different geometrical parameters, such as pre-crack position and length. The computational method presented in this study can be implemented to any PD constitutive model like Bond-Based PD or State-Based PD as well as any horizon factor. Several enhancements like cyclical annealing schedule and deformation gradient aware optimization technique are proposed to ensure the model would not get stuck in its trivial solution. The model’s performance assessment is conducted by monitoring the behavior of loss function throughout the training process. The results obtained through PD-INN demonstrate a satisfactory level of agreement in 2D and 3D when compared with the ground truth deformation from high-fidelity techniques like PD direct numerical method and Extended finite element method (X-FEM). Our results show the ability of the nonlocal PD-INN to predict damage and crack propagation accurately and efficiently.

Implementation of IDEAL algorithm based on Delaunay triangular mesh for 2D-compressible flows

PurposeTo present the detailed implementation processes of the IDEAL algorithm for two-dimensional compressible flows based on Delaunay triangular mesh, and compare the performance of the SIMPLE and IDEAL algorithms for solving compressible problems. What’s more, the implementation processes of Delaunay mesh generation and derivation of the pressure correction equation are also introduced.Design/methodology/approachProgramming completely in C++.FindingsFive compressible examples are used to test the SIMPLE and IDEAL algorithms, and the comparison with measurement data shows good agreement. The IDEAL algorithm has much better performance in both convergence rate and stability over the SIMPLE algorithm.Originality/valueThe detail solution procedure of implementing the IDEAL algorithm for compressible flows based on Delaunay triangular mesh is presented in this work, seemingly first in the literature.

A linearly implicit finite element full-discretization scheme for SPDEs with nonglobally Lipschitz coefficients

Abstract The present article deals with strong approximations of additive noise driven stochastic partial differential equations (SPDEs) with nonglobally Lipschitz nonlinearity in a bounded domain $ \mathcal{D} \in{\mathbb{R}}^{d}$, $ d \leq 3$. As the first contribution, we establish the well-posedness and regularity of the considered SPDEs in space dimension $d \le 3$, under more relaxed assumptions on the stochastic convolution. This improves relevant results in the literature and covers both the space-time white noise ($d=1$) and the trace-class noises ($\text{Tr} (Q) &amp;lt; \infty $) in multiple dimensions $d=2,3$. Such an improvement is achieved based on a key perturbation estimate for a perturbed PDE, with the aid of which we prove the convergence and uniform regularity of a spectral approximation of the SPDEs and thus get the improved regularity results. The second contribution of the paper is to propose and analyze a spatio-temporal discretization of the SPDEs, by incorporating a standard finite element method in space and a linearly implicit nonlinearity-tamed Euler method for the temporal discretization. The proposed time-stepping scheme is linearly implicit and does not suffer from solving nonlinear algebra equations as the backward Euler scheme does. Based on the improved regularity results, we recover the expected strong convergence rates of the fully discrete scheme and reveal how the convergence rates rely on the regularity of the noise process. In particular, a classical convergence rate of order $O(h^{2} +\tau )$ can be obtained even in high dimension $d=3$, as the driven noise is of trace class and satisfies certain regularity assumptions. The optimal error estimates turn out to be challenging and face some essential difficulties when the tamed time-stepping scheme meets the finite element spatial discretization, particularly in the context of low regularity and multiple dimensions $d \le 3$. Some highly nontrivial arguments are introduced to overcome the difficulties. Finally, numerical examples corroborate the claimed strong orders of convergence.

On the weak rate of convergence for the Euler–Maruyama scheme with Hölder drift

We consider stochastic differential equations with bounded α-Hölder continuous drift, with α∈(0,1), driven by multiplicative noise. We moreover assume that the diffusion coefficient is uniformly elliptic and bounded, with bounded first and second order partial derivatives. We show that the weak rate of convergence for the Euler–Maruyama scheme is arbitrarily close to (1+α)/2.

Dynamic optimization of nonlinear differential game problems using orthogonal collocation with analytical sensitivities

AbstractThis article presents an effective computational method based on the orthogonal collocation on finite element for nonlinear pursuit‐evasion differential game problems. The original problems are transformed into two dynamic optimization problems at first, so that the difficulty of obtaining the solution is reduced. To improve the convergence rate and the efficiency, the sensitivities describing the influence of control and interval parameters on state are derived through the discretized dynamic equations for the resulting nonlinear programming problem. The convergence speed is introduced to measure the performance in the upper level iteration. The main structure and the algorithm of the method are also given. Two demonstrative differential game problems with different scenarios from practice are studied. Compared with the approach without sensitivity information, the proposed method needs less function evaluations and saves at least 68.4% of the computational time. The research results show the effectiveness of proposed approach.

A sampling greedy average regularized Kaczmarz method for tensor recovery

AbstractRecently, a regularized Kaczmarz method has been proposed to solve tensor recovery problems. In this article, we propose a sampling greedy average regularized Kaczmarz method. This method can be viewed as a block or mini‐batch version of the regularized Kaczmarz method, which is based on averaging several regularized Kaczmarz steps with a constant or adaptive extrapolated step size. Also, it is equipped with a sampling greedy strategy to select the working tensor slices from the sensing tensor. We prove that our new method converges linearly in expectation and show that the sampling greedy strategy can exhibit an accelerated convergence rate compared to the random sampling strategy. Numerical experiments are carried out to show the feasibility and efficiency of our new method on various signal/image recovery problems, including sparse signal recovery, image inpainting, and image deconvolution.